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ABSTRACT 

This report consists priicarily of an executive 
suaaary of the findings of a study undertaken froa July 1974 to 
Septeaber 1975 by Kirschner Associates, Inc., for the Office of Child 
Development, Departaent of Health, Education and Welfare, in order to 
assess the overall effectiveness 6f the Project Head Start Training 
and Technical Assistance Prograa (T/TA). The study addresses itsalf 
to the following questions: Are the Head Start objectives 
foraulations, policy and guidance deveiopaent, need assessaent 
processes, T/TA planning, provider selection process, quality 
controls and the evaluation systea all appropriate and effective? How 
effective is the T/TA delivery systea? What is the quality of T/TA 
and what are its effects? Also included: a brief explanation of the 
Methodology of the study, a shprt discussion of the advantages and 
disadvantages of directly funding local prograas so they can purchase 
their own T/TA, and recoaaendations for change or iaproveaent in all 
the aforementioned areas. (MS) 
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INTRODUCTION 



Volume II consists of tvo parts: 



• an Executive Summary of the Findings and 
Conclusions contained in Chapter III of 
Volume I as well as a brief explanation 
of the methodology of the study, which 
is detailed in Chapter II of Volume I 

• the full text of Chapter IV in Volume I, 
Recommendations 
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EXECUTIVE SUMMARY 



I. INTRODUCTION 

This study was undertaken from July 197^ to September 1975 by 
fbv'Sckner Associates, Inc. for the Off ice of Child Development, 
Department of Health, Education, and Welfare, Washington, D.C., under 
Contract Na. HEW- 1 05-7^-1 1 12. The purpose of the proj.ect was to assess 
the overall effectiveness of the Project Head Start Training and 
Technical Assistance (T/TA) Program. The project scope was defined 
by the following set of topical questions: 

Statement of Purpose of the Project 
(in terms of T/TA topics and questions) 

M. Management of T/TA 

' Ml. Are appropriate^ and effective Head Start objectives formulated? 
M2. Is appropria^fe^and effective policy and guidance developed? 
M3. Are appropriate and effective processes followed to assess 

needs and devise T/TA plans accordingly? 
M^. Is an appropriate and effective T/TA provider selection 

process in place? 

M5. Are appropriate and effective quality controls exercised, ^ j 

e.g., reporting and monitorU^g? 
M6. Is an appropriate and effecxive evaluation system being 

Implemented? 

D. Delivery of T/TA , • ' 

Dl. How satisfied are the consumers with T/TA dollars available? 

D2. How effectively are resources used in T/TA service delive^ry? 

03. How effectively are other supportive resources being utilized? 

0^. How equitably is T/TA distributed among target groups? 

D5. How effectively are content arejps being covered? 

D6. How effectively are special content areas, i.e., nutrition, 

psychological services, and handicapped needs, being addressed? 



E. Excellence of T/TA 

EK Is the T/TA of high quality? 

E2. What effects does the T/TA bring about? 



n^c 



Special Section 

DF. Are there advantages to directly-funding local programs so 
that they can purchase their own T/TA? 
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One additional question was developed during the study concerning 
the issues of ''direct funding'* of local' grantees to permit their own 
purchase of T/TA services; this question was also addressed. Answering 
these questions encompassed the requi rements of the contract and, ^ 
therefore, created the basis for the project. 

II, METHODOLOGY 

In order to carry out this study, KAI staff devised a methodology 
approach Involving a comparative analysis on the topical questions of 
the national, regional, and local levels of Head Start J/tK activity 
-* ' (vertical) and then a comparison on each level of the viewpoints of ' 
both consumers .^nd provi^ders of T/TA (horizontal). This approach in- 
volved the interviewing of OCD Headquarters and Regional Office person- 
nel and, at the local level, directors, staff, parents and communii;ty 
leaders associated with sampled programs. Simultaneously, at each 
level providers of T/TA were interviewed'. A totaT of nearly 1,000 , 
persona! interviews were conducted. 

/ ' 

At the national level, Z'kOCD officials and 3h T/TA provider, 
staff (from nine organizations) were sim]ed. At the r^egional level, 
64 OCD staff (an average of almost sVk ior -each regional office) and 
119 T/TA sprovider staff were survey/d. (The '^provjder staff consisted 
of both pi^oject directors and staff and came from two different sample groups. 
Group one consisted of k2 represe/ntati ves of the RTO/STO network / 
interv/ewed in ten regions ; Group Two was comprised of 77 various regional 
providers, representing private contractors, universities, as v;.e11 as, 
again, the RTO/STO network, who were interviewed in the seven case study 
regions only. Those constycuting Group Two' were selected because of their 
provision of T/TA to they{)cal programs that were part of this T/TA eval- 
uation). At the local level ,\ programs were chosen to be part of the sample 
according to pre-determi ned criteria. A first cut at selection yielded 
70 programs. After conducting telephone interviews with the program oi rectors 
of each of these, a second cut at selection was made, resulting in the 
final 30 local programs that w3re spread across the seven case study regions 
(each of which had been chosen as components of a pur^posive sample repre- 
senting the various types of regional T/TA systems). Then hZQ directors, 
staff and parents, 162 community leaders, and 2k local providers- were inter- 
viewed. ^ 

ERIC ' ^ \ ' 
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Schematicai ]y , this comparative analysis approach, wit'o both i/c-rJ-icaJ 
and horizontal dimensions, would appear as follows. (Numbers within paren- 
theses represent interviews conducted.) 



OCD HQ ^ ^ National Providers 




X 



QCD RO < / — ^Regional Providers 
m / ^ (119) 



Programs (30) ^ > Local Prov<d<s»-s 

(660) (24) 



,The data collection plan for this approach was carried out by 
KAI Project Staff using field-tested structured interview guides and 
quest ionn|i ires. In addition, KAI staff studied numerous policy state- 
ments /froilv OCD -Headquarters, Regional Office T/TA Plans, and other 
(Jocuments* Wh^j;e appropriate, information from these sources-was 
integrated with tJiat data collected through interviews. However, 
the major foundation on which the study was stVuctured was data 
collected by the ihterviewfng process. ; 



III. FINDINGS AND CONCLUSIONS 

The findings and conclusions of this study are presented here ia 
^ the same sequence of topical questions listed above, i.e., according 

to the three major subjects of management, delivery, and excellence 
of Head Start training and technical assistance* 

Hanageipent of T/TA 

Ml . The annual ^eacy Start natic^al objectives exert -signif leant 

Impact on the T/TA program. This is especially true since the annual 
T/TA planning process at the national,^ regional, and to a lesser exfent,' 
local levels, addresses the requirements of these objectives. In / 



/ 
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examining the process of formulating national objectives, we concentrate^ 
ofi interviev/ing senior officials at the national and regional OCO,^ 
offices, including those persons directly inyoived in tnansginc T/TA 
aciivity (e,g*. CDTA personnel at Headquarters or PR6R Specialists 
Irt the Regional GfficesK We found chat great confusion exists among 
these staff as to who "is involved in establishmq these objectives and 
what process Is follcwc-d in their formulating* We could not identify 
a prescribed formal proces^s for this activity, but did find ad hoc 
mechanisms which operate through a variety of informal information 
networks. ^ _ [ 

Apparently, there is no clear mechanism for incorporating input 
from the local level into the process of national objective formulation. 
This lack inhibits the National Office from formulating new objectives 
for local programs that would be appropriate f6r national focus and also 
from knowing the local ^capacity to absorb new thrusts before current 
ones have been achieved. We encountered considerable sentiment for in-^ 
creasing the amount of local level input. 

Another issue concerning the formulation of national objectives 
that surfaced was the problem of timing, i.e», frequently objectives 
are established too late in the year to be efficiently and effectively 
integrated .Into T/TA planning process. Regional Office personnel 
interviewed were especially concerned about/this issue in that their 
J/TA plans must be begun each year before they know definitely the 
national objectives. 

We asked NationaTand Regior^l Office personnel to rate their 
success in achieving the FY 7^ objectives. Those objectives which 
were rated as being'most successfully achieved, e*g., 10% enrollment 
opportunity foV the handicapped, and compliance with the performance 
standards, were also the ones most clearly understood and backed by 
well-defined policy statements. This fact suggests that T^TAwill 
have its biggest impact in implementing national objectives when those 
objectives are well thought-out and effectively articulated. 
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Conversely, if the objective^ are not we 1 1 -conceived and communicated 
then T/TA wijl not be of much help in accomplishing them, 

H2. The training and technical assistance program is dependent 

on pol icy and guidance issued by OCO because it both defines the 
situational context in which T/TA Is delivered and the actual T/TA 
activity. An example js the T/TA Planning Guidance for Regional 
Offices chat is prepared each year by lihe National Office, in years 
pest, the guidance has required excessive amounts of information 
from the regions. Apparently, this burden on regional T/TA personnel 
was a means by which the OCO Headquarters collected data on the national 
status of Head Start; the various T/TA planning systems v;ere called' 
upon to fill the void caused by the lack of a comprehensive national 
data collection system. The FY *76 Planning Policy simplified the 
requirements for the Annual Regional Office T/TA Plan. These 
streamlined requirements for^ the Plans seem to solve many of the pro- 
blems in the process, e.g., level of specificity and formating of 
information, that were discovered in our interviewing Regional Office 
staff. One other related problem was that of timing. Frequently, 
the instructions for Regional T/TA Plans come out before the national 
objectives have been finalized, thereby making it difficjult for the 
regions to develop their Plans in an efficient and effective w^ay. 

Another dimension of the relationship betv/een Headquarters -and 
the Regional Offices on this subject that we noted was a feeling ex- 
pressed by some central office officials that re^4..20^l staff may not 
be doing a thorough planning job. This finding is noteworthy in that the 
planning process was supposedly decentralized a couple of years ago 
and still the national officials are reacting as though the planning 
authority has not truly been delegated. This phenomenon is perceived 
by regional personnel and probably accounts, at least in part, for 
their resistance to the planning requirements, it is very plausible 



/ 
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that this tension ri creating good effects, however, because as will 
be discussed subsequently in this Summary, there are abundant indications 
that good T/TA is being delivered to grantees through the regional 
provider network. 

StiU another problem involving . the T/TA planning processes 
mandated by OCD policy and guidance for the Regional Offices is that of 
insufficient manpower. Regional staff believe they do not have 
the number and kind of staff resources to respond to the National 
Office request for T/TA Plans in the time frame allowed. This shortage 
becomes even more critical when Regional Offices attempt to meet other, 

often concurrent, needs imposed by the central office. • 

"I 

The orse example of policy and guidance considered here, that 
involving T/TA planning, ms the most pertinent one as^ far as our 
evaluation was concerned. Other policy and guidance issues that we 
,enountered were a lack definition of compliance with the 
performance standards, and such things as nepotism, facility repairs, 
s etc. If the long-awaited revised Head Start Manual could be finalized, 
presumably these oth^r issues would be addressed. 

H5, The issues of the Annual T/TA Pj^n, which each Regional Office 

must submit leads to another, broader issue about which KAI uncovered 
information— the coordination between and among staff and providers 
at the national, regional, and local levels regarding T/TA planning. 
National Office respondents were conscious of the tension that has 
surfaced in the past several years between them and Regional Office 
staff. Th^ Headquarters officials, on a five-point scale ^^excellent, 
very good; good, fair, and poor," tended to rate their coordinatiorf 
with the Regional Offices as very good, while the regional personnel, 
in turn, rateds the ccondinaticn betv/^en themselves and Headquarters 
qenerally as faV. Conversely, the kegional Officials tended to rate 
their coordination with the local programs as very good, whereas of 
the local program personnel only a bare majority (53. 7^^) rated the 
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coordination between themsel ves' and their Regional Office positively 
(i.e., excellent, very good, or good,)- This phenomenon is probably 
explainable to some degree by managers believing they are doing an 
effective job (perhaps more effective than may be the case) of managing 
and by those being managed not giving ,suffic lent credit to those 
responsible for managing them. Granting either explanation to be 
applicable, the. conclusion can still be drawn that that National Office 
apparently can do better vis-a-vis the Regional Offices and the 
Regional Offices apparently can do better vis-a-vis the local programs 
in terms of coordination for T/TA planning. 

The national, regional, and local pioviders all gave themselves 
very positive marks (i.e., excellent or very good) on coordination with 
the appropriate national, recjional, and local level Head Start people. 
In order to get a second indi^Lator about coordination regarding 
T/TA' panning, all the providers were asked if they had to submit a 
workplan to their respective /national , regional, or local employer. 
Answering ^^yes^* were €>k.7% of the national providers, 93.0^ of the 
RTO/STO network, 92.2^ of the various regional providers, and 97.5^ ^ 
of the local providers. These percentages tend to confirm the testi- 
mony of the providers that they coordinate very well with their employer 
on T/TA planning. The only notable exception is with the national 
providers, less than two-thirds of whom said they had an obligation 
to submit a T/TA workplan to the central office. 

All respondents in our study were asked to rate the effectiveness 
of the T/TA needs assessment and planning process (needs assessment 
being ^an Integra 1 and essential part of any T/TA planning). At the national 
level, OCD respondlnts generally reported their role as being limited 
to after-the-fact collection of data from the regional and lt)cal 
sources and to a quality-control function. They collectively tended to 
say that the processes at the regional and local levels were only *'fair,'* 
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The national providers, on tt^'^'^^her hand, reported having a much 
larger rol^/in these processes; and their rating of process effective- 
ness was 79.5^ positive (i.e., excelleint, very good, or good). A 
dramatic divergence of opinion regarding T/TA needs assessment and 
planning functions thus emerges between OCP Headquarters staff and 
national providers. 

Regional Office interviewees usu=»l]/ reported their roles in these 
processes as primary; and their ratings of process effectiveness were 
81.0^ positive. The various regional providers generally said that 
their role in needs assessment and T/TA planning was also primary or 
critical; and their ratings of the process effectiveness were 93.6% 
positive. These findings show a much closer perception of things by 
staff and providers at the regional level than was manifest at the 
national level. They al^o indicate that regional providers interviewed 
tended td rate the effectiveness of needs assessmentx and planning for 
T/TA more highly than Regional Office staff. By yirtue of their roles, 
regional providers probably have a greater stake jin giving favorable 
answers. 

, At the local level ,\ those directors, staff, and parents surveyed 
tended to describe their role in these processes as critical; their 
ratings on process effectiveness were 77.3% positive (excellent, 
very good, good)- By comparison, the local providers also tended to 
count their role in these needs assessment and T/TA planning processes 
as critical; and their ratings were 70.8% positive, somewhat lov^er than 
those given by respondents directly associated with the local programs 
sampled, yet still indics^tive of a rather closely parallel viewpoint. 
Interestingly, regional providers' ratings of the processes were higher 
than regional staff, whereas local providers* ratings were lower 
than local program respondents. lr\ both cases, however, the differentia 
seems mrnor compared to that observed at the national level between 
OCD officials and T/TA providers. 

8 ' 
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4i(^^rmation was also gathered on thB criteria used in the assessing 
and prioritizing of needs for T/TA planning. Performance standards 
and national objectives were rt'cntioned rrtost often by national providers 
and Regional Office staff; regional and^cat objectives, by the 
various regional providers; and parent, staff,, and community needs, 
as well as local objectives, by both the local provider and program 
respoj^dents* At the local level less emphasis was noticed on natiwal ^- 
or regional objectives and more emphasis was seen on the needs of 

parents, st^ff, and the community as well as on local objectives. 

/ 

} 

I ^ 

M^. ^ Because provider selection is another indicator of T/TA management, 

KAI staff sought data on this Issue. National and. Regional Office staff 

\ - , 

detailed the process of contracts and grants utilized for provider i 

selection and collectively rated the selection processes quite positively! 

80.0^ (National Office) and 88.8^^ (Regional /Of f ice) . Regional res- | 

pondents were somewhat more positive than National ones on this item. 

A J. 

At the local level, one of the most-eignif icant findings was that 
approximately 70.0^ of all those interviewed were unaware of how their 
Regional Off ices^elected T/TA providers. The. size of this majority 
suggests that local interviewees are accorded' little chance for input 
into the selection process for providers that will serve them ultimately. 
This finding may help explain why a bare majority (53.7?) of local 
respondents rated the cooperation betvieen their Regional Offices and 
themselves positively (i,e., excellent, very good, good). Among 
those local level persons who were familiar with the process employed 
by the Regional Office, 83*0? rated it posit\yely, More importantly, 
a bi-variate analysis showed that those local level respondents familiar 
with how the Regional Office chooses its providers tended to be more 
satisfied overall with their T/TA than those unfamiliar with Regional 
Office processes. 



/ 
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H5 The next topic addressed was control of providers through report- 

ing, monitoring, and coordinating mechanisms after they have been selected. 
At the national level, tho OCD staff interviewed generally reported that their 
•processes for controlling T/TA providers were very good to excellent* How- 
ever, data obtained from the providers themselves showed that less than two- 
thirds of them (at most) submitted an initial T/TA work plan and only about 
half of them reported on their work progress to the National Office. 

At the regional level 95.0^ of .the members of the RTO/STO network wl^o were 
.interviewed said they submitted reports on their T/TA activity to their Re- 
gional Office, as did about 70.0^ of the various regional providers sampled. 
Couple this with the other finding, cited earlier, that 93.0^ and 92.0% of 
these two groups of regional providers, respectively, submitted initial T/TA 
work plans, and it becomes evident that, there seems to be considerably more 
provider control exercised at the regional level than at the national level. 

The picture changes dramatically again, however, when focusing on the 
local level. Only one-half of the local program personnel interviewed said 
they submitted any kind of progress report and,, of those, only 17.0% specified to 
the Regional Office, Only 75.0% of the local providers said they submi tted^ 
any kind of report (vs. 91.2% of national providers and 90.0% (approximately) for 
regional providers) and only ^2.0% specified to the Regional Office. It is, of 
cojrse, plausible to bay that local program personnel and providers, particularly 
those who are direct-funded and therefore are responsible for buying their own 
T/TA, need not necassarily report to their Regional Office about their T/TA ac- 
tivity. It is also important to point out that the data suggests that local 
program personnel and providers do quite a bit of reporting to others besides 
the Regional Office, e.g., grantee boards. Neither of these factors entirely 
wipe away the impression created by these data, hov/ever, that local T/TA ap- 
parently is not as closely controlled as regional according to our samples. 

All things considered, Jthe contrjol of p-oviders as evidenced by procedures 
such as work plans and reporting seen^s to be strongest at the regional level. 



Also, across the national, regional. 



andl local level?, there seems to be no 
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great consistency of reporting procedures or f requencles.j This finding leads 
to the Issue of monitoring. Few, if any, regions seemed to have in place a 
nx)nItoring system that would integrate with and feed into a comprehensive man- 
agement information system (MIS) that »^uld benefit both the individual regions 
and the central office by providing current data on T/T/i accompHshments,/activ- 
ities, and needs. This type of a system, implemented on an on-going basis, 
•^uld greatly facilitate the planring processes for T/TA each year and would 
allow Head Start officials to have available to them at any time a ''state of 
the system" retrieval. 

Given the data on how stringently the regions control their providers 
by means of work plans and reports (compared to the national and local 
levejs), many regions obviously have the makings of such a management infor- 
mation system. Wnat is lacking is a unity of all the actual and potential 
parts from the individual regions. This issue, therefore, , seems to be one 
that requires great attention for the overall operations of the Head Start 
T/TA program! to, improve. 

M6^ After considering how T/TA is controlled during the period of its 

being given to consumers, KAI then focused on how it is evaluated. The note 
struck above about lack of consistency^ (in regard to reporting procedures) 
is the appropriate one to center on again by way of introducing this topic. 
On all three levels, national, regional, and local, there seemed to be widely 
\ divergent practices regardirrg evaluation . 

i 

On the national level, OCD Headquarters respondents generally reported that 
there" was no uniform evaluation system in place and that as a result individual 
project officers or divisibns (e.g., CDTA, PDel) determined what sort of evalua- 
tion would be carried out. National providers ten<^d to confirm that this is 
^ the case. A very large number of them, however, (9^,0%) do*evaluate their work 

in one way or another, whether or not they are obligated to conform to a par- 
^ticular set of evaluation requirements by OCD Headquarters. 

At the regional level, 80.0^ of the Regional Office staff interviewed re- 
ported that some sort of evaluation system exists, but, as was true on the na- 
tional level, the system is not uniform from one region to the next. , Over 90.05$ 

14 
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of all regional providers sampled (i.e*, 97. 0^^ of the RTO/STO network members 
and 32.2% of the various regional providers) reported carrying out some sort 
of evaluation, be it through reports of the consumers, third-party observation, 
etc* One noteworthy piece of information uncovered on this subject was that 
87.0% of the regional providers sampled conduct written evaluations, as opposed 
to only 53.0% of the national providers. This information is an additional in- 
dication that regional providers are more closely managedT^ha^i^ thei r national 
counterparts. ^ 

As far as the local level is concerned, Sh.0% of the pi^p/am people and 
75.0^ of the providers said they carry out some sort of evaluation. Only A7.0% 
of the local providers said they submitted written evaluj^tibns; in contrast 
with the regional level, evaluation at the local level tends to be more verbal 
than written. There also seems to be much the same diversity of evaluation ap- 
proach at the local level as was found nationally. A bivariate analysis of 
local level evaluation data showed that those program personnel who carry ojut 
an evaluation tend to be more satisfied with and perceive a greater impact from 
their total T/TA received than those who do not evaluate their T/TA. 

In short, most respondents carried out some I<ind of evaluation, but there 
apparently is no clear-cut overall system in place at either the national level, 
across all regions, or at the local level that integrates all the discrete evalu- 
ation efforts. ' ^ 

Del i very of T/TA 

Dl . Initially, on the subject of delivery of T/TA, the persons interviewed 

at the local program sites were asked whether or not they were satisfied with the 
T/TA dollars available to them. The majority said they were' not satisfied and 
80,0% of them reported that they would like more T/TA mo^ey if they could have 
it. This data suggests an underlying sentiment that T/TA being delivered to 
local program respondents is probably beneficial and hence they wish it were 
possible to get more in order to address their other current unmet T/TA needs. 
Of the 30 programs sampled, nine received direct-funding monies (PA 20) to purchase 
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their Gwn T/TA. The other 21 programs visited v/ere asked this same question 
but answered it from a more indirect perspective, i.e., as recipients of T/TA 
from natio-.al, regional, and state providers offered to then. 

D2. To get more specific information on the current status of the T/TA 

delivery system, topics on present T/TA benefits and unmet T/TA needs were 
addressed. Data was gathered on exactly what resources are ut r 1 ized by the 
local programs in order to get T/TA. The findings were generally that 
the grantees sampled tend to get equal parts of their total T/TA from national, 
regional, and non-Head Start sources.. The fact thac there seems to be a rather 
balanced distribution of T/TA from these three sources is in itself noteworthy. 
What is also notable is that local programs seem to receive as much T/TA from 
non-Head Start sources as they do from either national or regional providers. 
This phenomenon suggests that apparently local programs are quite resourceful 
in soliciting free T/TA services from their community resources and that com- 
munity agencies or organizations are quite willing to donate their expertise 
to Head Start grantees. It is a most encouraging finding. 

Provicfers at all three levels were asked about resources they had avail- 
able and hoy they used them. Resources as a term here in the context of pro- 
viders is being used differently than it was in the context of Head Start staff 
T/TA providers are a resource for Head Start staff. But proyiders also have 
their own resources from which to draw, e.g., how they organize their time to 
serve the Client, specific areas of expertise they have, etc. Regarding the 
use of time, 32.0^ of the national providers, 53-0% of the regional, and 17.0% 
- of the local reported that they followed some pre-set formula so as to utilize 
better their time resources. Regarding their subject-mattier . experti se, the 
categories of T/TA they reported providing most frequently across all three 
levels were education, parent involvement, and, handicapped. Performance stan- 
dards T/TA was mentioned very often on both the national and regional levels. 
More '/ill be said on content areas of T/TA below (D5 and 06). 
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D3, Supportive , or alternative, resources for T/TA was another topic on 

which data was collected, Regarding the support and cooperation from local 
community leaders on T/TA matters, 60 .0.^ of the local program people interviewed 
rated it very positively (a great deal and quite a bit), as did 70.0% of the 
leaders themselves. One-half of the community leaders sampled reported having 
some official connection with their local Head Start, e.g., serving on the 
Policy Advisory Council. Both of these findings reinforce that reported in 
D2, namely, that local grantees are receiving large amounts of T/TA help from 
their community leaders and the organizations which they lead. A bivariate 
analysis of these data showed that the more help a respondent reported re- 
ceiving from supportive resources, ^.g. , community agencies, the more satisfied 
with overall T/TA received that person tended to be. Similarly, the impact . 
perceived from all T/TA was greater among those who received more help from / 
supportive resources than those who did not. A corollary to this finding is ' 
that if a local grantee is active in soliciting T/TA from supportive community 
resources, that grantee is probably more knowledgeable about T/TA matters gen- 
erally and therefore more appreciative of any additional (donated) T/TA ser- 
vice. Very possibly these same grantees are good managers of T/TA from any 
source. 

On the subject of parents as a supportive resource in T/TA matters for a 
local Head Start program, 53.0^ of the local program people and 56.0% of the 
community leaders interviewed rated their support and cooperation very posi- 
tively (i.e., either excellent or very good). This rating is down somewhat 
from that given to the corrmiunity leaders, but still shows how much help apparent! 
* is given by parents to local programs on T/TA matters-. The rating also suggests 
on-going success is being achieved by Head Start in its mandate to involve par- 
ents in its programs. 

04. The amount of T/TA to various target groups , i.e., administrators, 

coordinator? , teachers, teacher aides, support staff, parents, and others in the 
local program, merited particular focus. On every level, national, regional, 
and local, and among bot! program people and providers interviewed, there Sur- 
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faced an ove™helming unanimity of opinion that all target groups need more 
T/TA. This consensJs of opinion was highest, however, for parents, coordinators, 

and administrators, and in that order. These data relate to and support those 
mentioned in Dl, namely, that the vast majority of local program interviewees 
want more T/TA money available. 

Local program directors cited staff turnover 85.0^ of the time as a factor 
necessitating some additional T/TA, This frequency, of course, has a positive 
\ aspect, i.e., presumably some of those staff who have previously benefitted from 
T/TA services are able to parlay them and their whole Head Start experience into 
,^ a better job opportunity. Our study yielded no data on this phenonmenon; we are 
only theorizing when , suggesting this explanation 

\ 

\ 

\ 

D5^ As mentioned earlier, content areas of T/TA would be more amply dis- 

cussed. Recall that the most frequently offered categories of T/TA were edu- 
cation, parent involvement, handicapped services, .and performance standards. 
Other data gathered showed the negative aspect of :his topic, i.e., those con- 
*:ent areas overlooked or Inadequately covered. Handicapped services T/TA was 
most frequently mentioned amdng all respondents as being inadequately covered, 
suggesting that even though it is a subject. being addressed mightily, it is still 
needs further attention. This situation has a positive aspect to it, in that 
obviously Head Start has succeeded in establishing its commitment to the handi- 
capped as a ve»-y high priority. Other content areas that were mentioned fre- 
quently as being overlooked or inadequately covered were management and admin- 
istration, parent education (vihich confirms the finding cited earlier that all 
respondents feel more T/TA for parents is needed), health services, social ser- 
vices, interpersonal or group dynamics, and child development/psychology. 

D6. Three special content categories of T/TA were examined somewhat more 

in-depth at the suggestion of National Headquarters officials: nutrition, . 
psychological services, and handicapped. Local program respondents were asked 
how much need they had for additional T/TA in these three areas. Seventy per- 
cent (70.0%) said either ''some, quite a bit, or a great deaP* of additional 
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T/TA was needed in each of the three areas. Consiifering just the two responses,- 
"quite a fait and a great d^al,*' more distinct variations appear, however: kS*B% 
gave those two answers fro handicapped T/TA still needed; ^3.0^ for psychological 
services T/TA; and 32.0? for nutrition T/TA* 

Regarding handicapped services T/TA, particular subcategories were men- 
tioned as requiring additional T/JA* They included general handicapped train- 
ing (working with handicapped; Identifying and screening handicapped; and de- 
veloping and getting resources and services), specific liandicapped training 
(tTiental retardation, learning disabilities, emotional disturbances, speech 
defects, physical handicaps, etc.) and needs' of others training (parents, staff, 
non-handicapped peers, community, etc.)* ' \ 

Specific resouces that could be tapped in .order to meet these needs, ac- 
cording to the local program interviewees, included, in order of frequency of 
response, community agencies, universities or colleges, and regional providers. 

Problems encountered jn trying to identify and incorporate handicapped 
chi Idren, into local Head Start programs grouped into these clusters: lack of 
support and supplies, untrained staff, parental difficulty in acceptance of the 
handicapped child, teaching conditions, and recruitment problems resulting from 
geographic isolation. All throughout this exam,ination of the special circum- 
stances surrounding handicapped services T/TA for Head Start, a constant re- 
frain was that additional T/TA is definitely needed in order to perform more 
effectively the task of serving the handicapped." 

\ ' ' ' ' 

Excellence of T/TA \ 

\ 

El . There are two indicators 9f excellence that we considered: quality 

and effect. Both >?ere given discrete consideration, on the theory that quality 
T/TA does not necessarily result in effective T/TA, and vice versa. One mea- 
sure of T/TA qual I ty utilized was the level of satisfaction with T/TA received. 
The overall level of satisfaction seemeo^'qnrtTe-^w-gi^ i.e., ndrmally in the 
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ninth or tenth deciles. i Considering *Very satisfied and satisfied^* responses 
together, regional providers had a 92.2% response rate; local providers, 87.51; 
national providers, 85.3^3; community leaders, 82.8^; and local program di- 
rectors, staff, and parents, S]:S%* 

The regional providers rated satisfaction with their own T/TA more highly 
than did the national or local providers. Granted the rating of satisfaction 
by providers themselves could be self-serving, it still is apparent that the 
regional providers tend to be the most satisfied. This collective perception 
corresponds to data presented earlier about T/TA management indicating that 
the Regional Offices seem to do a good job of managing their providers. In 
other words, good management is resulting in high satisfaction when the T/TA 
is final ly del ivered. 

A second* measure of quality of T/TA was in certain key elements of T/TA 
presentation, i.e.^ preparation, familiarity with Head Start purposes and needs, 
appropriateness of presentation, thoroughness of subject knowledge, ability to 
meet needs of and communicate with participants, appropriate materials and tech- 
nlques, follow-up, evaluation, and sensitivity to needs of the poor. National 
providers tended to rate themselves "normaV* on all elements, using a prede- 
termined rating system (^,3,2, or 1, ^ being the best); regional providers 
tended to rate themselves above the "norm" on all elements; and local providers, 
like their-national counterparts, usually scored around, the Vnorm." / 



Bivariate analysis of these data from local program respondents showed 
that there is a positive relationship between high ratings on these elements 
and high satisfaction (very satisfied) with overall T/TA. ^ Conversely, the 
lower the ratings given on these key elements by a given respondent, the more 
dissatisfied that same respondent tended to be overall- with T/TA. 



Local program people rated the^r satisfaction wij:h T/TA in three special 




categories as follows: for nutrition T/TA, 69.9? very satisfied/satisfied; 
handicapped services T/TA, 58.2^; and for psychological serviced T/TA, 55*3%* 
These percentages indicate that the latter two categories are in need of more at- 
tention than the first. 
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On the matter of satisfaction with various sources of T/TA, these same 
local level respondents reported being satisfied or very satisfied in the' 
following frequencies: • , 

national providers 76.5^ 

regional providers 83.8% 

locaf providers (PA 20 fupds) 95.5^ 

local providers (program funds) 90,9% 

non-Head Start sources 93.8% 

These figures show that the closer to the local level the source of T/TA is, 
the greater the percentage of respondents expressing satisfaction. Also they 

show that non-Head Start T/TA sources are providing highly satisfactory 
services. 




E2. Another component of excellence is T/TA effect . A measure of effect ^ 

was the amount of impact perceived by various respondents. The overall level 
of perceived impact seemed quite high. Considering the answers *'a great /deal, - 
quite a bit, and some'* (on a scale of "a great deal, quite a bit, some* a little 
or none), the frequencies of response for. each category of respondents were ^ 
these: 32.3% of the regional providers; 85*0^ of the local program people; , 79.2% 
of local providers, and 76.5^ of national^ providers. As was -the case with satis- 
faction ratings (£1), regional providers rated thejnselves more highly in terms 
of T/TA impact than did national or local providers. Qnce more, the data indi- , 
eating that regional providers are^well rnanaged comes to mind when examining^ 
these findings on impact. There seems to be a constant pattern throughout the 
topics addressed in. this evaluation suggesting that the best'managed Head Start 
providers are on the regional level and now another noteworty corollary can be 
found here, i*e., they are also the group of providers that collectively per- 
ceive their impact to be the greatest as a result of T/TA they deliver. 

A second measure of effect of T/TA w^s in certain key characteristics, of 
the delivery of T/TA, i.e., completeness, practicality, informativeness, and 
timeliness. Considering national, regional, and locals providers, more respon- 
dents. in each category gave high ratings to two characteristics, practicality 
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and informati veness, than to the other two characteristics, completeness and 
timeliness. These ratings Indicate areas in v^hich T/TA may be^ improved to 
strengthen Impact. If the T/TA can be made more complete and timely, then 
greater Impact is likely to occur. The regional providers, following what 
by now is'a very familiar pattern, tended to rate their T/TA much more favorably 
than either the national or local providers rated theirs. 

Both' the local program pebple and the community leaders consistently ^^ve 
similar answers— sti 1 1 another, albeit very^ Indirect, indication of^the ap- 
parent close relationship between the two groups on matters involving T/TA. 
More importantly, tf^eir answers confirm the view of all provider respondents 
that complete and timely T/TA is more difficult to deliver, but when accom- 
plishied results in greater impact on th'e program. 

Another noteworthy finding an'rong these data was that, as a rule, T/TA de- 
livered po parents—as opposed to either the staff or the program considered as 
a wholg~was rated lower in terms of all^four key characteristics^ This infor- 
mat ion relates closely, to that revealed above in D4 about how parents were 
regularly perc^elved to be a target group that needed more T/TA. 



A bivariate analysis of these data showed that there is indeed a pos^itive 

relationship between high ratings on these four key characteristics. .and high 

Impact perceived result|ng from overall T/TA, 

1 



DF. Several factors comparing di rect-funded programs to ^non-di rect-funded 

programs were examined. On the matter of satisfaction with overall T/TA re- 
ceived there seemed to be no notable differences between respondents from the 
hine'dlrect-funde4 programs and those from the other 2^^ programs in our sample. 
However, on the matter of impact perceived from T/TA, more respondents associ- 
ated with direct-funded programs {70. k%) reported that the impact of T/TA was 
great than did those- associated with non-direct-funded programs (53*S^)- This, 
of course, seems to be the most important place to look for a comparison, I.e., 
the effect, that T/TA is having. The data suggest strongly that a greyer ef- 
fect results from T/TA that is directly purchased by a local program. This 
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finding relates closely to thdt in El showing that the closer to the local 
level tl^ source of the T/TA, the greater the chances for satisfaction with 
overall T/TA by local program- people. 

Similarr results were obtained, when analysis of other measures of T/TA 
effect, i.e,, the key characteristics described in E2 (complete, practical, 
informative, timely), was undertaken. 
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IV, RECOMMENDATIONS 

Up to this point in this Final Report, KAl has presented the data collect- 
ed on T/TA and isolated for the benefit of the reader the salient findings 
and conclusions that surfaced in the data. The thrust our efforts in 
Chapter III was to be objective, ite,, to display the data and let it speak 
for itself, fn this Chapter, we are presenting for consideration by OCD 
officials some pertinent ^econmendations we believe can justifiably be made 
in light of the findings and conclusions. These recommendations are subjec- 
tive on our part and may not be the same ones other readers of the data would 
make,. We offer them, however, as interpretive judgments v;hich OCD can consider 
in determining their future T/TA management activities. 

First, there is much about which Project Head Start should be elated* 
For Example, its local programs (judging by our sample) seem to be doing a 
phenomenal job of getting T/TA services donated by their cofununity resources, 
This finding suggests a very^-^trong impact has been made by local programs on 
their communities { as was verified by KAl five years ago in its N ational 
Survey of the Impacts of Head Start Cdriters on Community Insitutions) and that 
a very positive and cooperative relationship exists between grantees and community 
resources. 

* , Second, regional providers of T/TA as a group (again, judging by our san- 
ple) seem to be well coordinated and delivering T/TA that is quite excellent. 
Discounting the fact that part of the basis for saying this is the testimony 
of regional providers themselves, it s^ill can be stated that the regional- level 
providers tended to compare very favor^ably to the national and local ones in 
terms of how they were managed, how they delivered T/TA and the excellence of 
the final product. This finding is one for which, no doubt, both the Mational 
and Regional Offices deserve a share of the credit, in spite of all the friction 
over the past several years regarding Regional T/TA Plans demanded by the 
Headquarters, the end result apparently has been a good one, 

The recommendations set forth here v;ill follow the basic topical format ; 
of Chapter I II on Findings and Conclusions* 
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HI , Regarding the setting of^ national obfectives It seems imperative 

that OCD HQ devise a mechanism that "institutionalizes*' or makes fore formal 
the processes for gathering input on and setting up annual objectives. This 
mechanism should then be articulated to both national and regional level per- 
sonnel so that they know exactly how to feed into the process if they so desire. 
Further refining and articulation of the mechanism should take place. in 
order to maximize local level input, This will ultimately aid the entire T/TA 
system, even though the inpact of such a recommendation obviously transcends 

T/TA matters. 

I 

H2, Numerous policy and gurdance issues were mentioned as needing updating 

or initial attention. Granted, much h^s been accomplished in the past several 
-years for which OCD HQ deserves enormous credit, but much still remains to be 
done. The Revised Head Start Manual seems to be grossly overdue; its publica- 
tion would presumably resolve many questions that currently are unansv;ered. 
Failing the publication of the Manual in the nea^ future, another solution to 
the problem of needed policies would be Individual issuances dri the most 
pressing subjects. ^ 

On the subject of the Annual Regional T/TA plan mandated by OCD Head- 
quarters, may problems cited by the'Regional Offices, e,g.. a new format 
every year, too much information required, etc, apparently have been alleviat- 
ed as a result of the FY 76 pqlicy issuance. It still remains to be seen, how- 
ever, whether the current policy will function v/ell over a period of years or 
whether it will become outmoded next year and need refinement or replacement* 
This uncertainty seems reasonable given the fact that Project Head Start is 
really only beg inning"^ the process of decentralizing its T/TA program and the 
transition period is bound to be one filled with tension. The current tension 
seems to be not entirely a bad thing, in as much as there is abundant evidence 
that the T/TA being delivered by the regional providers as a consequence of the 
Regional T/TA Plans is quite effective, Therefore our recommendation v/ould be 
not to discount the value of the T/TA planning processes that have been carried 
out by the regions over the past several years in complinace with National 
Office mandates. The national OCD ought to pursue even more diligently 

25 • ■ 

(568) 



KIRSCHNER ASSOCIATES INC. 



\ a balance that allows the OCD Headquarters to coordinate all T/TA In accord 
with established goals and objectives, and, at the same timej enables each 
individual Region to plan its own T/TA program with the maximum amount of 
autonomy. 

JiL As far as needs assessment and T/TA planning is concerned, it 

seems chat the National office should reconsider its role in these matters. 
Is its role only secondary or after-the-fact as many central office staff 
mamtained? What needs are being addressed when national providers are 
hired? On what basis have those needs been detennin^? Why do national pro" 
viders tend to feel much, more involved in these needs assessment and T/TA 
planning processes than do OCD officials? Even if OCo's role is to be only 
secondary, i.e., to collect data- gat he red at the regional and local levels, 
what mechanisms does i t have in place to 'integrate . these data and make, appro- / 
priate decisions for the total T/TA program based on the data? KAI v^uld 
recommendthat these kinds of questions be addressed by OCD officials as it 
reconsiders rnle in needs assessrnent and T/TA planning processes, KAl 
would. also recommend that a comprehensive Management Information System (MIS) 
be considered as one possible mechanism to aid in the overall tasks of needs 
assessment and T/TA planning, 

There appears to be concurrence at both the regional and local levels 
among program perspnnel and providers as to what their roles are vis-a-vis 
-needs assessment and how well those roles are being fulfilled. What remains to 
be done, however, is to refine and perfect the tools used foY the assessing 
of needs. Apparently a number of exca,llent as'sessment tools are available, 
such as those, among others, in Regions 11, iV.^and X, But whether or not 
they are all readily available to interested, part ies is dubious. Even if the 
various tools were used only as guidance resources, a wider distribution of them 
would be very beneficial, 

gisyjder^s elect ion is a process that needs regular reexamination to 
prevent complacency among Head Start staff regarding the providers they are 
utilizing to deliver T/TA. Constant refinfng and Improving of the criteria ' 
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fallowed for select'on purposes and continual building of safeguards to preserve 
a proce-is that encourages the choosing of providers nrast able to meet the specific 
needs of a given. consumer group are two obvious things that can be recommended 
in this area. 

Local level respondents seemed woefully unaware (70^ of them) of the 
procedures followed by their Regional Offices in selecting providers to serve 
their region. It would seem advantageous for the Regional Offices to not only 
apprise their local programs of their provider selection procedures, but also to' in- 
vite them to participate in them as appropriate. Recall that a positive 
reiationship was found to exist between f ami 1 iari ty with the regional selection 
processes and overall satisfaction with T/TA. 

M5, The essential recommendation regarding provider control is serious 

consideration of a comprehensive management information system that would, 
integrate data on all, facets of the T/TA program at the national, regional, 
and loca'1 levels. Such a system would provide for retrieval of data at any 
given time to meet a multitude of needs, from satisfying Congressional 
information requests to preparing for the formulation of national Head Start 
objectives; from the coordi nating.of needs assessment and T/TA planning act- 
ivities at the national, regiori'al and local levels to the tracking of monitoring, 
reporting, and evaluation results from various T/TA activities. 

This recommendation need not lead back to centralized control of the nearly 
$20 million annual T/TA budget at OCD Headquarters. On the contrary, it would 
enable the central office to coordinate the total T/TA program more effectively, 
the regions, to continue to plan and implement their own T/TA programs autonomously, 
and the local programs to operate more efficiently. 

Another recommendation on this topic is that OCD HQ. devise a more coordinat- 
ed system for control of the national providers, Steps in the direction of a 
better system might well include more standard reporting and evaluation mechanisms 
for project officers who act as liaiscn.to national providers, and better inte- 
gration of efforts carried out by separate divisions, e.g., POSI, CDTA, PMD, in 
relation to nationol provider control, 
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tJii Concerning evaluation of T/TA, our basic recommendation is to do 

more of it. Granted, this effort would require time and money, however, 
additional evaluation would force the T/TA managers, providers and recipients 
to be more concerned about the itnpact of T/TA, 

At the national level there appears to be no uniform system for a) 
evaluating national providers; or b) collectinq evaluation deta on regional 
and local providers. We suggest that responsibility in the central office for 
both these activities be assi^gned clearly. Those responsible should devise 
adequate policy and guidance covering evaluation procedures and then articulate 
those procedures effectively to the involved personnel at the national, region- 
al, and local levels. 

J" 

DK Regarding satisfaction with T/TA dollars ^ all categories of ^ 

respondents manifested considerable dissatisfaction. There are valid reasons 
for this. Our study uncovered strong data showing that certain target groups 
are not getting sufficient T/TA and that specific content areas are not being 
adequately addressed. Accordingly v/e recommend that OCD officials qive consid- 
eration to possible ways to provide additional financial support for the Head 
Start T/TA program, 

D2. On th3 topic of utilization of resources in matters relating to T/TA, 

KAMs study uncovered evidence of strengths in all facets of the delivery sys- 
tem employed by national, regional, state, and local Head Start personnel. 
Therefore, we do not intend to recommend that "any one piece of the overall de- ^ 
livery system be scuttled. On the contrary, the various national, regional, 
state, and local T/TA resources on the whole serve discrete and valid functions 
and tend to complement me another. 

Our finding on this topic of resource utilization should be reiterated 
here: the closer the source of T/TA is to the local level, the greater 
the satisfaction with overall T/TA by local program people. This finding leads 
us to reco.Timend that OCD continue to provide mechanisms, such as direct-funding, 
which will enable local programs to have effective access to T/TA resources. 
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Still another finding of note was that local programs sampled seem to have 
a good Balance of T/TA services fron national, regional, state, local, and 
non-Head Start resources. We c?te this as further buttressing for our suggestion 
that all levels of the overall T/TA delivery system for Project Head Start 
be maintained. However too much emphasis on any one resource, to the extent that 
other T/TA resources are minimized, does not at all seem justified according 
to our data. 

03. As mentioned in the introduction to his chapter local programs 

apparently ar«j doing a phenomenal job of getting T/TA services donated by 
supportive resources , such as their community agencies and organizations. We 
recommend that OCD officials look into this phenomenon further and investigate 
possible ways to further capitalize on the apparent rapport existing 
at the local level between the grantees and their communities. 

Our data on this topic also shov/ed that local Head Start programs are 
receiving substantial help in T/TA matters from the parents of enrolled 
children. We therefore recomnend that OCD officials also look for additional 
ways to capitalize on the resources of parents - a suggestion certainly in 
line with Head Start's committment to parent involvement. 



oil. In regard to the topic of target qrouns, D=irents are again the appro- 

priate focus of di cuss ion, because they ranked first as a category of persons 
in need of additional T/TA. They were followed, in order, by coordinators and 
administrators. All target groups moreover (i.e., the above three, plus teachers 
aides, and support staff) were clearly perceived by our respondents to need 
additional T/TA; no target group was reported to need less T/TA. Therefore 
we recotmiend that Head Sta-t officials and staff work to devise ways to improve 
the coverage of T/TA across all appropriate target groups. 



05. Rp.aardinQ content categories of T/TA KA! collected data showing that 

the most frequently offered categories were education, parent involvement, 
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handicapped sen/ices, and performance standards. We also found that handicapped 
services T/TA was most frequently mentioned as an inadequately covered category, 
indicating that even though it is a subject being addressed nightily, it 
Still needs further emphasis. Other content categories mentioned often as being 
either totally overlooked or inadequately covered were nanagement and adninistra- 
tioni parent education (which parallels the finding cite-* -^rl i«r about parents 
as a target group needing more T/TA), health services, social services, inter- 
personal or group dynamics, perfornance standards, and child development/ 
psychology. We recorrmend to OCD, as a result, that appropriate steps be taken 
to fill these gaps in T/TA content categories. 



D6. 



So ^^i" as special categories of T/TA are concerned, i.e., nutrition. 



psychological services, and handicapped, KAI gave special emphasis to these 
in response to suggestions from OCD HO, officials. The data indicate t.hat a 
high need exists for T/TA in these three special areas and that the need is, 
comparatively, most for handicapped (which parallels the finding cited in the 
previous section} next for psychological services, and last for nutrition services, 
One-half to one-third of our sample perceived their needs as high in each of 
these three special categories. Accordingly KAI recommends to OCD that an effort 
be made £6 fill these special unmet needs/ 

^ Data from the topics relating to T/TA excellence (quality and impact) 

have been used to support recommendations made about the T/TA rnanage- 
ment and delivery systems. No specific recommendations on these topics will 
be made. 



OEjl. On the topic of direct funding of local programs to purchase their 

own T/TA, KAI's study uncovered strong evidence showing a positive relationship 
between direct-funding and perceived impact of overall T/TA. This finding indi- 
cates it has been very effective. Any decision to expand the practice of 
direct-funding,, however, necessarily must take into consideration other issues, 
some of which have been touched on in this evaluation and some of which have not. 
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A local grantee must have the capacity Both to assess needs and to plan for 
a T/TA program. It mast further possess corrmensurate skills In other areas 
of T/TA managementt \t nust, for example, be able to identify and recruit 
appropriate T/TA resources that fit Its financial framework and fulfill its 
T/TA needt". The implication here is that grantees will always need more JjTA 
than is available to them via di rect-^fund ing, Grantees must be able to depend 
upon state, regional and national resources for T/TA services. 

It is also safe to assume that ir. some cases direct-funding for purchase 
of T/TA is not going to be the most cost-efficient. This situation would likely 
exist in rural areas or with regard to highly specialized forns of T/TA, In 
cases such as these, a broader state and/or regional system seems appropriate.. 
Hence KAf recommends that OCD retain the direct-funding mechanism as a valid 
and viable option for delivering T/TA but be judicious in the selectihg'of 
the option. 



Lastly, we have a recofrnendation on the dissemination of this Final 
Report, Since this document contains, but does not isolate for ease of re- 
view, much data on strengths and weaknesses of the seven case study regions, we 
suggest that a~ series of subsidiary reports be prepared, one for each individual 
case study region, utilizing data contained in this Report that is presently 
> organized by topic and not by region. This dissemination procedure would help 
the regional office staff focus on the findings directly relevant to their 
operation. We also suggest that ways be considered to disseminate the results 
of this study to, the local program level as well. 
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